Random forest versus logistic regression: A large-scale benchmark experiment

BACKGROUND AND GOAL The Random Forest (RF) algorithm for regression and classification has considerably gained popularity since its introduction in 2001. Meanwhile, it has grown to a standard classification approach competing with logistic regression in many innovation-friendly scientific fields. RESULTS In this context, we present a large scale benchmarking experiment based on 243 real datasets comparing the prediction performance of the original version of RF with default parameters and LR as binary classification tools. Most importantly, the design of our benchmark experiment is inspired from clinical trial methodology, thus avoiding common pitfalls and major sources of biases. CONCLUSION RF performed better than LR according to the considered accuracy measured in approximately 69% of the datasets. The mean difference between RF and LR was 0.029 (95%-CI =0.022,0.038) for the accuracy, 0.041 (95{\%}-CI =0.031,0.053) for the Area Under the Curve, and - 0.027 (95{\%}-CI =-0.034,-0.021) for the Brier score, all measures thus suggesting a significantly better performance of RF. As a side-result of our benchmarking experiment, we observed that the results were noticeably dependent on the inclusion criteria used to select the example datasets, thus emphasizing the importance of clear statements regarding this dataset selection process. We also stress that neutral studies similar to ours, based on a high number of datasets and carefully designed, will be necessary in the future to evaluate further variants, implementations or parameters of random forests which may yield improved accuracy compared to the original version with default values

Random forest versus logistic regression: a large-scale benchmark experiment

Results on partial dependence. Additional file 3 includes a study on interesting extreme cases that allows to gain more insight into the behaviour of LR and RF using partial dependence plots defined in “Partial dependence plots” section. (PDF 256 kb

Additional file 2 of Random forest versus logistic regression: a large-scale benchmark experiment

Datasets from biosciences/medicine. Additional file 2 presents the modified versions of Figs. 3, 5 and 6 as well as Table 2 obtained using the datasets from biosciences/medicine only. (PDF 288 kb

Additional file 3 of Random forest versus logistic regression: a large-scale benchmark experiment

Results on partial dependence. Additional file 3 includes a study on interesting extreme cases that allows to gain more insight into the behaviour of LR and RF using partial dependence plots defined in “Partial dependence plots” section. (PDF 256 kb

Additional file 1 of Random forest versus logistic regression: a large-scale benchmark experiment

Additional results of subgroup analyses. Additional file 1 extends Fig. 5 for all considered measures, and include the outliers. (PDF 203 kb